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Problem solving is a critical element of learning physics. However, traditional instruction often 
emphasizes the quantitative aspects of problem solving such as equations and mathematical procedures 
rather than qualitative analysis for selecting appropriate concepts and principles. This study describes the 
development and evaluation of an instructional approach called Conceptual Problem Solving (CPS) which 
guides students to identify principles, justify their use, and plan their solution in writing before solving a 
problem. The CPS approach was implemented by high school physics teachers at three schools for major 
theorems and conservation laws in mechanics and CPS-taught classes were compared to control classes 
taught using traditional problem solving methods. Information about the teachers’ implementation of the 
approach was gathered from classroom observations and interviews, and the effectiveness of the approach 
was evaluated from a series of written assessments. Results indicated that teachers found CPS easy to 
integrate into their curricula, students engaged in classroom discussions and produced problem solutions of 
a higher quality than before, and students scored higher on conceptual and problem solving measures. 
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I. INTRODUCTION 

Physics teaching in both high school and college places 
an emphasis on problem solving [1-8], and although 
students demonstrate reasonable competence in traditional 
assessments of problem solving skills, there is evidence that 
understanding of fairly fundamental concepts is weak or 
lacking following completion of introductory courses 
[9-14]. Students in introductory physics courses solve 
problems largely using a process termed means-ends 
analysis, whereby they search for equations containing 
the quantities in a problem and try to reduce the “distance” 
between the goal state and their current state in the solution 
process [5,8,15,16]. Students are not taught to solve 
problems simply by manipulating equations since instruc¬ 
tors typically mention the concepts and principles that they 
are applying, but students rightly perceive the equations 
as being central to obtaining quantitative answers and tend 
to ignore conceptual information. This approach can be 
effective at getting answers, but falls short in understanding 
the conceptual underpinnings of the solution process. It is, 
therefore, not surprising that students learn or retain little 
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conceptual knowledge following introductory physics 
courses. 

Although physics instructors at all levels would agree that 
integrating conceptual knowledge with problem solving is a 
desirable goal in physics instruction, traditional materials 
tend to promote, albeit inadvertently, equation manipulation 
at the expense of conceptual understanding. Standard 
physics textbooks present equations in terms of general 
symbols and elaborate upon what those symbols stand for; 
however, there is little guidance for students regarding when 
it is useful to apply a particular relation to a problem [15]. 
Understanding the “conditions of applicability” for a prin¬ 
ciple and the procedures for determining whether the 
necessary conditions have been met are essential for 
proficient problem solving, and these conceptual aspects 
need to be made explicit during instruction [6,14,17]. 

The equation-centered approach favored by beginners is 
in contrast to the more strategic approach favored by skilled 
problem solvers. Skilled problem solvers organize their 
solution strategies around major principle(s) or concept(s) 
[2,5,18]. For skilled problem solvers, principles or concepts 
in memory are also bundled with contexts or conditions in 
which they can be applied and with procedures for applying 
them [2,15,19]. This type of integration of major ideas, 
contexts, and procedures provides skilled solvers with a 
hierarchically structured, well-integrated knowledge base 
that guides their problem solving [17,18,20]. Since build¬ 
ing such a knowledge base takes considerable time and 
effort, an important consideration is whether there is benefit 
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in attempting to help students in introductory courses 
develop the type of knowledge needed for skilled problem 
solving. Several studies suggest an affirmative answer. 

One intervention in introductory mechanics with under¬ 
graduates began with a conceptual overview of the material 
in qualitative form [21]. Students learned to reason about 
phenomena qualitatively, using various representations 
(e.g., free-body diagrams and energy bar charts) for the 
concepts learned. After students learned qualitative ways of 
describing and reasoning about physical phenomena, they 
then experienced a second pass at the same content but this 
time with mathematical descriptions. They also worked on 
multistep problems that required application of knowledge 
from different parts of the course, thus integrating knowl¬ 
edge across course components. Additional activities 
included problem categorization, where students identified 
the major principle(s) or concept(s) needed to solve prob¬ 
lems without actually solving them. Students experiencing 
this intervention outperformed students in a traditional 
course in conceptual and problem solving assessments, 
and were also better able to retain conceptual knowledge 
long after the course was over. The same results were 
replicated with at-risk students at a different university [22]. 

Another attempt to integrate conceptual knowledge within 
problem solving used a computer-based tool that allowed 
users to analyze problems in terms of the principles or 
concepts needed to solve them [23,24]. To analyze problems 
using the tool, students made selections from a series of 
menus, with each subsequent menu becoming increasingly 
specific and building on choices made in previous menus. 
By selecting appropriate principle(s) or concept(s) (e.g., 
work-energy theorem or conservation of energy), and 
specifying initial conditions and problem context (e.g., 
whether or not frictional forces were present, whether a 
body possessed kinetic and/or potential energies in some 
initial state), conceptual qualitative analyses of problems 
could be performed, which in turn resulted in one or more 
equations that the tool generated to fit the specific analysis 
performed. Compared to students who experienced a more 
traditional control treatment, students who used the analysis 
tool were better able to categorize problems according to the 
underlying principle [25] and to solve problems. 

Yet another study [26] conducted in a large introductory 
mechanics college course asked that students write 
conceptual, qualitative “strategies” [descriptions of the 
principle(s) needed to solve a problem, a justification for 
why the principle(s) applied to the problem, and a procedure 
for applying the principle(s)] for problems in homework and 
exams prior to generating a solution. Course insUuctors 
modeled writing quality strategies in lecture and discussion 
sections, and posted solutions for the weekly homework 
assignments that displayed both strategies and solutions for 
all problems. When compared with students undergoing 
traditional insUuction, students who practiced strategy writ¬ 
ing displayed benefits in conceptual measures, including 


better ability to categorize problems according to principles, 
and better ability to identify the major ideas covered in the 
course several months later. 

At the high school level, studies of the impact of 
integrating conceptual knowledge in problem solving on 
students’ conceptual understanding are rare, although there 
is a curriculum [27] that targets the development of physics 
conceptual knowledge, and a program [28] specifically 
aimed at helping students overcome misconceptions. 
However, both are designed in a context outside of problem 
solving and with no mathematics. One high school study 
[29] did compare the impact of “explicit problem solving 
instruction” to traditional problem solving instruction on 
various conceptual and problem solving measures. Both 
approaches included five steps: 

• Explicit problem solving: (a) focus the problem; 
(b) describe the physics; (c) plan the solution; 
(d) execute the plan; and (e) evaluate the solution. 

• Traditional problem solving: (a) Draw a sketch; 
(b) define known and unknown quantities; (c) select 
equations; (d) solve equations; and (e) check the 
answer. 

The step of “focus the problem” guides students to state a 
general approach in terms of concepts and principles, 
however, it does not emphasize the justification for the 
appropriateness of those principles in the manner that the 
Conceptual Problem Solving method does [30,31]. 
Although there were some advantages for the explicit 
problem solving group, the two groups performed equiv¬ 
alently in tasks requiring conceptual knowledge, such as 
planning solutions and conceptual questions. 

The current study investigated a question with dual 
goals: Can pedagogical approaches that have been shown 
to be effective at promoting CPS among college students be 
adapted for use in a HS setting in ways that allow flexible 
adoption by teachers, and that promote measurable positive 
outcomes in students? The need for flexibility in adoption 
among high school teachers is important. The college 
studies reviewed above that promote conceptual develop¬ 
ment within problem solving were both developed and 
implemented by the same faculty member or researcher, 
thus creating investment or ownership as well as insuring 
fidelity of implementation. On the other hand, it is 
unrealistic to have high school instructors adopt wholesale 
an approach handed to them by university researchers. 
High school teachers face constraints and challenges in 
teaching their physics courses that may place limits on the 
time they devote to implementing the new approach, as 
well as to the style in which they implement it; for example, 
the time that can be devoted to implement a new approach 
as well as to administer assessments to measure outcomes 
was limited. It was impossible, therefore, to design a 
classroom-based high school study in which the teachers 
adopted or delivered the approach in carefully controlled 
ways that one could achieve in the researchers’ laboratory. 
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The intervention was designed to provide teachers with 
ways of highlighting the role of conceptual knowledge in 
problem solving while keeping the implementation of the 
intervention flexible, thereby allowing teachers ways of 
promoting conceptual problem solving in ways to fit their 
curricular demands. Materials were developed to encourage 
students to perform qualitative analyses of problems 
(similar to the strategies described in Ref. [26]), and then 
to use those strategies in formulating quantitative solutions 
to problems. The intervention attempted to steer students’ 
tendencies away from equation hunting and toward follow¬ 
ing a general framework for solving problems that began 
with identifying principles to apply, justifying why they 
could be applied to the problem, and then generating a 
solution plan to generate an answer. 

The second goal was equally important, namely, meas¬ 
uring whether the approach had an impact on student 
outcomes. Assessments needed to be developed that 
attempted to measure conceptual understanding as well 
as problem solving. The assessments developed took more 
time to administer than teachers typically had available to 
devote to noncourse-related assessments. Teachers had the 
flexibility to decide how much class time they would 
devote to administering study assessments subject to their 
curricular constraints, as well as to which particular assess¬ 
ments would be administered. 

There were additional challenges facing the study. 
Whereas almost all the studies cited above evaluating 
interventions to promote conceptual problem solving were 
done with college students who had declared an interest in 
pursuing a STEM major, the high school students in our 
study had much more diverse interests and mathematical 


preparations. In addition, unlike the college studies cited 
earlier where those designing and implementing the inter¬ 
vention were physicists, the high school teachers in this 
study reflected the background of teachers nationwide— 
some were teaching out of field while others had degrees in 
physics. Finally, the number of students participating in 
each class in our study was small given physics enrollments 
at the participating high schools and the high schools 
were different enough that we felt it best to analyze each 
separately, placing constraints on the interpretation of the 
findings. Even though this limitation requires a focus on 
trends across schools, we believe the trends are fairly 
strong. In summary, the study reported here was challeng¬ 
ing, looking for what were likely to be somewhat subtle 
changes in problem solving and conceptual understanding 
resulting from an intervention that was implemented in 
different ways over different durations by teachers with 
varying degrees of physics competence to students with 
varying degrees of physics interest and mathematical 
preparation. 

The next section describes the materials, participants, 
and experiment design in more detail. It is then followed by 
the study results and a general discussion of the findings. 

II. METHOD 
A. Materials 

1. Learning materials 

Conceptual Problem Solving (CPS) is not a curriculum, 
but rather is a framework for solving physics problems that 
can be easily adapted to existing course materials. The 
features of CPS are adapted from Ref. [26] and modified 


Problem: At point A on a roller-coaster, a 150 kg car is traveling at 13 m/s and 
is 3 m above the ground. Calculate the speed of the car at point B, when it is 
5 m above the ground. 


Principle: 

Conservation of energy: the total mechanical energy of an isolated system (sum of 
kinetic and potential energies) is the same in the initial and final states. 

Justification: 

There are no non-conservative forces doing work on the roller coaster car as it 
travels along the track (we neglect air drag and friction). Therefore no energy is 
gained or lost and the energy of the car at point A is equal to the energy of the car 
at point B. 

Plan: 

1. Draw a picture and assign symbols for quantities in the problem. Choose a 
coordinate system. 

2. Write an equation for conservation of mechanical energy. Expand this 
equation to include the initial and final kinetic and potential energy terms. 

3. Solve for the final speed of the roller coaster car. Substitute values and 
calculate a numerical answer. 

FIG. 1. Sample problem and strategy (principle-justification-plan). 
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FIG. 2. Sample two-column solution. 


to be more suitable to a high school student population. 
The strategies used in Ref. [26] for the study with university 
STEM majors were intended to be prose descriptions of the 
principle(s), justification(s), and procedure(s) needed to 
solve problems, but no specific guidance was provided for 
how to write strategies other than telling them that a good 
strategy had to be a logical discussion of how to solve a 
problem that contained the three pieces above. For the high 
school student population we broke down this process into 
more specific stages to provide more guidance to students 
on what was expected of them. Further, because writing 
qualitative descriptions for solving problems is a high level, 
difficult task, we deemed it important to provide as much 
structure as possible to help students succeed in writing 
strategies, while at the same time not being overly 
burdensome in our requirements. The CPS approach that 
we finally decided upon contained three separate parts: 
Principle (the principle or concept applicable to the 
problem), justification (explanation of why the concept 
or principle is appropriate), and plan (numbered steps that 
provide the “recipe” for solving the problem and the 
equations that do with each step in the plan. Students 


carry out their plan for solving the problem by formatting 
their solution as two columns: A left column describing the 
plan step, and a right column consisting of equations or 
mathematics that go along with the step [32]. When the 
method is first introduced, scaffolding is provided in the 
form of headings or blank worksheet templates that prompt 
students to complete each part. A sample strategy is 
presented in Fig. 1 and its accompanying two-column 
solution is presented in Fig. 2. 

It is important to note that students were not required to 
write the principle, justification, and plan before carrying 
out the quantitative solution on the right-hand side of the 
two-column solution, although most students did, as did 
instructors when illustrating the approach to students. Here, 
as in Ref. [26], the aim was to highlight the role of 
conceptual knowledge in problem solving, and this could 
be accomplished by writing the principle, justification, 
and plan either before or after generating a solution with 
equations. 

A sample set of problems, strategies, two-column 
solutions, and blank worksheet templates were written 
by the authors prior to classroom implementation of 
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CPS. Typically 10 or more problems were developed for 
each physics topic, exhibiting a range in problem 
complexity and difficulty. The topics included motion in 
one and two dimensions, Newton’s laws, momentum 
(Impulse-Momentum Theorem and Conservation of 
Momentum), and energy (Work-Energy Theorem and 
Conservation of Energy). Upon request from participating 
teachers, a few problems were also written for minor topics 
including vectors and torque. The files for these problems 
and solutions were provided to teachers participating in the 
study, who were given the option to use these problems as 
written, modify these problems, or to use their own problems 
but format them to look similar to the samples. 

2. Assessments 

Students from the traditionally taught classes and the 
CPS classes completed a series of written assessments to 
evaluate their understanding of physics concepts and their 
problem solving skills (see Supplemental Material [33] for 
sample items from the assessments). There were five 
different tests administered to assess different aspects of 
the students’ understanding, which we briefly explain here. 
Three of the tests (problem solving, conceptual questions, 
and categorization) examined important aspects of what 
might be learned from CPS, another test examined whether 
there was any influence on a different complex skill 
(finding errors), and the final test examined students’ 
understanding of the variables in the final equation (equa¬ 
tion instantiation). Examples of each of the tests are 
provided in the Supplemental Material [33] and should 
be examined when reading these explanations. We return to 
these assessments again in the results section. 

Problem solving .—Items in this test are standard free 
response problems, similar to traditional problem solving 
measures used in physics courses, that students were asked 
to solve. Most physics courses use problem solving as their 
primary assessment, so it was deemed critical to examine 
problem solving. CPS is meant to help the students to 
proceed from a problem to a principle, so should improve 
problem solving. Total time: 15 min for 3 items. 

Conceptual questions. —Items in this test prompt stu¬ 
dents to write free-response explanations for a realistic 
physical situation (school A) or answer conceptual ques¬ 
tions in a multiple choice format (schools B-C). This test 
evaluated whether students could link physical situations to 
principles and concepts—a basic goal of CPS. Total time: 
15 min for 6 items. 

Problem categorization. —Items in this test are similar to 
the 3-problem categorization tasks used in Ref. [25] but 
modified to be grade appropriate. Students must select one 
of two problems that would be solved most like a third 
“model” problem. Problems are specifically designed to 
vary their match on superficial features (objects or context) 
or match on concepts and principles used to solve them. 
This test evaluated ability to determine if problems were 


solved with similar principles and whether students were 
distracted by the presence of similar surface attributes in 
problems. Total time: 25 min for 9 items. 

Finding errors .—Items in this test contain a problem 
statement and worked-out solution that includes a con¬ 
ceptual physics error. Students are asked to identify and 
describe the error in writing, and are specifically informed 
that there are no mathematical errors in the solutions. This 
test measured if students could detect solutions that were 
lacking the application of a needed principle or solutions 
that applied concepts incorrectly. Total time: 15 min for 
3 items. 

Equation instantiation .—Items in this test include a 
problem and worked solution in symbolic form. Students 
are asked to assign or match the appropriate values for each 
quantity in the final expression. Because the final expres¬ 
sion from the principle is given, it does not require any 
decision about the correct principle or justification for it. 
Instead, the test requires students to be able to match 
particular quantities in the problem to variables in the final 
expression. This is important knowledge for getting the 
correct answer, but it is not what would be learned by CPS, 
and we did not expect this would lead to much difference 
between the groups. It is possible that there will be some 
small effect because a better understanding of the principles 
could also lead to better understanding of what the principle 
variables map onto in a problem. Total time: 15 min for 
5 items. 

B. Participants 

Physics teachers from four high schools agreed to 
participate in the study, however due to unforeseen circum¬ 
stances that affected the implementation and assessment 
procedures, one of the schools was excluded from the data 
analysis. 1 The remaining schools and CPS teachers will be 
referred to as A, B, and C. These teachers were selected in 
part because they were teaching nonadvanced placement 
classes at their school and had some flexibility in designing 
classroom activities. 

The particular experiment design at each school was 
determined by situational constraints, such as the number 
of physics teachers and number of classes offered there. At 
every school there was at least one CPS class and one class 
taught using traditional, equation-focused problem solving 
methods. At school A there was one teacher who taught the 
same course over multiple years, at school B there was a 
single teacher teaching two sections of the same class using 
different methods, and at school C there were two different 
teachers. Because of such vast differences in the design, the 


1 Despite a previous verbal agreement, the control teacher at this 
school was unwilling to administer the assessments during class 
time. This restriction of CPS students to ones willing to come to 
an out-of-class testing session means the sample was susceptible 
to a potential bias, so we excluded the data. 
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TABLE I. 

Profile for each high school participating in the study. 




School 

Design 

Teacher experience 

Location 

Low income 13 

AYP 2b 

School A 

One teacher, multiple years 

In field, Experienced 

Suburban 

12.0% 

68.8% 

School B 

One teacher, multiple sections 

Out-of-field, Inexperienced 

Rural 

53.2% 

47.2% 

School C 

Two teachers 

Both in field. Both experienced 

Small city 

31.5% 

60.0% 


“Proportion of students at the school classified as low income. 

b Proportion of students who meet or exceed federally mandated standards for adequate yearly progress (AYP) in reading and 
mathematics. 


student populations at each school, and the teachers’ 
knowledge and experience teaching physics, each school 
will be treated separately in the data analysis. 

A brief profile for each of the four schools is provided 
in the Table I below. Information regarding the proportion 
of students who are classified as low income, and the 
proportion of students who meet federally mandated 
standards for AYP (adequate yearly progress) in reading 
and mathematics are taken from public reports by the 
Illinois State Board of Education [34]. 

School A is a small suburban school with an experienced 
physics teacher and an affluent, high-achieving student 
population. This was the teacher’s second year participating 
in the project, with the first year used as a control class and 
taught without CPS. All students were in their senior year. 
School B is a rural high school with a student population 
that is primarily low income. The teacher taught two 
sections of the same course, one using the CPS approach 
and one using his usual way of teaching problem solving 
that emphasizes equations and mathematical procedures. 
The teacher’s area of expertise is chemistry, and this was 
only his second time teaching physics. Students enrolled in 
this physics course at school B are in their junior year. 
School C is a high school in a small city with a diverse 
student population. There were two teachers both teaching 
different sections of the same course (one used CPS and 
one taught traditionally). Although both teachers have a 
strong background in physics the CPS section teacher 
typically teaches math courses and the control teacher has 
far more experience in terms of years teaching physics. 

All participants whose data are reported signed consent 
forms allowing their results to be used (students under 
18 years of age signed assent forms and their parents signed 
consent forms). 

C. Procedure 

1. Training procedure 

Teachers met for a half day with researchers during the 
summer prior to using the CPS approach to receive minimal 
training and guidelines for implementing Conceptual 
Problem Solving. The teachers at each of the three schools 
were given guidance on key elements of the CPS approach, 
such as the goal being to focus on the role of concepts 
in solving problems. They were shown specific examples 


of several problems solved using the CPS approach 
(see Figs. 1 and 2). Teachers were also told that they 
would be given detailed CPS solutions to problems (as in 
Figs. 1 and 2) during the semester as they covered new 
material to use in their teaching. We pointed out to them 
that they had substantial freedom to adapt the CPS 
approach to their particular class and student population. 
This was intentional since no curricular intervention will be 
implemented unless teachers have the flexibility to use it in 
ways that fill their instructional needs. Prior to this short 
training session, the teachers were not familiar with the 
approach, other than previously agreeing to participate in 
the study prior to the training. Our observations suggested 
that the teachers bought into the CPS approach to the extent 
that they were presented with previous research evidence 
favoring conceptual approaches to problem solving (as 
reviewed in the introduction), and to the extent that they 
found the approach reasonable and useful for classroom 
implementation. 

2. Observations and debriefing meetings 

Each CPS teacher’s classroom was observed at least 
once for each major physics topic (four times) during the 
implementation. Each teacher met with the researchers for a 
debriefing session at the conclusion of the study; these 
semistructured interviews were audio recorded, which 
provided self-reported descriptions of how they imple¬ 
mented CPS, what they thought about teaching this way, 
how frequently they used materials, and teachers’ percep¬ 
tions of students’ reactions to the approach. 

3. Implementation of the intervention 

The conceptual problem solving approach was designed 
to be implemented during classroom activities that per¬ 
tained to problem solving instruction. Since the emphasis 
placed on problem solving varied for each course and 
student population, the time spent on CPS also varied. 
Other course activities such as lecturing, demonstrations, 
and laboratories remained unchanged. 

There was substantial flexibility in how teachers could 
implement the approach, and as a result the fidelity of 
implementation varied across teachers. There were varia¬ 
tions across teachers in how much time they spent on the 
approach during class, how classroom activities were 
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structured, and how they assessed student understanding of 
the approach. Despite these differences, there were some 
common elements across all of the schools. Each teacher 
devoted at least two class periods to the approach for each 
2-3 week long topic unit studied. They were all accus¬ 
tomed to having students work together in cooperative 
groups of between two and four students during class, and 
CPS was no exception to this procedure. All of the teachers 
chose to grade students on their use of the approach in some 
fashion, and typically this occurred on an exam. How 
teachers implemented the approach was in fact one of the 
factors of interest and so we report below on our obser¬ 
vations regarding implementation as well as teachers’ 
descriptions of their own implementation of the CPS 
approach. 

Implementation at school A. Teacher A was an experi¬ 
enced physics teacher who preferred to adapt the approach 
to his own established set of homework problems. He 
reported implementing the conceptual framework primarily 
for more advanced problems, which represented between 
one-third and one-fourth of students’ homework (problems 
with multiple steps, multiple principles, or more advanced 
mathematical procedures). Students were not graded on 
homework, however, at least one problem on each exam 
required students to use the approach. He was observed to 
model the strategy-writing aspects for students when doing 
example problems on the board during class (principle- 
justification-plan), but only used the two-column solution 
in an informal way. 

For example, late in the semester teacher A was observed 
to use the approach for a complex problem in which a block 
slides down a frictionless curved ramp, collides with a 
second block, and the blocks fall off of a table onto the floor 
below. He prompted students to decompose the problem 
into three segments and identify a relevant principle for 
each part (conservation of energy, conservation of momen¬ 
tum, and motion with constant acceleration), which he 
wrote on the board. The Justifications for these principles 
were brief statements: “frictionless ramp,” “elastic colli¬ 
sion,” and “constant acceleration.” He then guided students 
to construct a plan of 3 steps and wrote them out in words 
on the board. Then he took each step individually and (with 
verbal prompts from the students) completed the math¬ 
ematical solution for each step sequentially. Although the 
solution was not presented side-by-side in the two-column 
format, all of the components were present. 

Teacher A also devoted some class time to “synthesize” 
the big ideas that had been covered during the fall semester. 
He did this by showing a diagram that listed the major 
theorems and conservation laws in boxes, and arrows that 
connected them to corresponding equations. He empha¬ 
sized to students that if they are uncertain how to begin 
solving a problem, they should refer back to the “big ideas” 
from the class and select one of these concept(s) or 
principle(s). 


Implementation at school B. In contrast, teacher B 
primarily teaches chemistry and physical science courses, 
and was teaching this physics course out of field for only 
the second time. As a result, he described his instructional 
plans as somewhat fluid and subject to change on a weekly 
basis. Teacher B taught one section of the course using the 
conceptual problem solving approach, and taught a second 
(control) section using a more equation-centered approach 
(a framework called GUESS—write what you are Given, 
the Unknown quantity, an Equation, Substitute values, and 
Solve.) He almost exclusively used sample problems 
provided by the researchers for both of his classes, but 
formatted them in a different way for the control class. 
When introducing the conceptual approach for a new topic, 
he was observed to write the strategy on the board word 
for word from the researchers’ examples and refer to the 
example on paper as he did so. Students in the CPS section 
were given blank worksheets that prompted them for each 
step in the strategy and to format their solutions as two 
columns, and they often worked in cooperative groups. 
Students’ homework was graded and at least one problem 
on each exam (for the CPS class) required them to complete 
a strategy and two-column solution. Teacher B described 
his classroom implementation of CPS in the following way: 

“Initially it was more my guidance, and then while they 
were still learning it, then they were in pairs or maybe 
even three to help each other out getting through it while 
I’d be going around. And then toward the end it was 
more individual. It’d be, ‘Ok, start on it individually, 
and then now let’s compare and see where we are. ’ And 
then go big group and I’d have somebody talk about 
what they thought” (teacher B). 

In addition, teacher B later adjusted the way worksheet 
templates were used in response to student feedback. When 
some students complained about writing their plan steps 
and then recopying them into the left column of the solution 
template, he modified the approach to have students write 
their plan steps directly into the left column of the two- 
column solution template rather than writing it as part of the 
strategy. 

Implementation at school C. Teacher C has a strong 
background in physics, but was hired primarily to teach 
math classes at his school and only taught physics classes 
on an “as needed” basis. He used many of the problems 
provided by researchers but also incorporated his own set of 
problems. He admitted that he did not use the approach 
very much for beginning topics like one-dimensional 
motion, but increased his use of strategy writing substan¬ 
tially for later topics like energy and momentum (approx¬ 
imately six times in a 3-4 week period). Teacher C felt that 
the approach was more relevant later in the semester: 

“I didn ’t focus a lot on it during the first half of 
the semester because the entire first half is constant 
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acceleration...I was kind of holding back on it until we 
had a few more things in our toolbox before we could 
start to use it to differentiate between the different 
concepts. So as we started to get into momentum and 
energy and torque and things like that I started to use it 
more frequently” (teacher C). 

At the introduction of a new topic, he would write a 
sample strategy and the initial plan steps on the board with 
verbal guidance from students. The plan step usually 
involved a general framework of drawing a picture, listing 
what you know, writing equations (separating the x and y 
directions when appropriate), and solving for the unknown 
quantity. The solve step was not very specific; he told 
students that as they got started the details would become 
more obvious. Students followed the two-column solution 
format on homework, but were not required to do so 
on exams. 

Teacher C made extensive use of cooperative group 
problem solving and categorization with the conceptual 
approach. During a group problem solving session, he 
handed out worksheet templates for four to six problems 
that prompted students to complete only the first steps of 
principle and justification. Then, each student group was 
required to write out a plan and solution to just one or two 
of those problems, which they presented to the class. 

4. Assessment procedures 

As described previously, there were five different written 
assessments (see sample questions provided in the 
Appendix). However, due to time constraints not every 
school gave every test. The researchers allocated time per 
test and all the tests were gauged to take 85 min (25 min for 
the categorization test and 15 min for each of the four 
remaining tests). However, teachers had different limita¬ 
tions on how much class time they could give. The total 
time allotted for the assessments ranged from 45 to 85 min 
(administered in either one session or split over two class 
periods). Teachers did not have access to the assessments 
ahead of time. The assessments completed by participating 
students were scored by researchers and did not contribute 
to students’ grades. 


Questions that required written explanations or problem 
solutions from students were scored independently by two 
researchers using agreed-upon rubrics (this pertained to the 
free-response conceptual questions used as school A, and 
problem solving questions used at all schools). These 
scores were further discussed by the researchers to reach 
a single consensus score. The problem solving questions 
were scored according to a rubric modeled after Ref. [35], 
which scores solutions on the categories of useful descrip¬ 
tion, selecting a relevant physics concept or principle, 
applying the concepts to the specific conditions in the 
problem, executing mathematical procedures, and the over¬ 
all communication of a logical reasoning pattern. The 
finding errors test was scored for both identifying a mistake 
in the sample solution and giving an appropriate explan¬ 
ation for the mistake. 

III. RESULTS AND DISCUSSION 
A. Assessment scores 
1. Description of assessment scores 

The average scores on each assessment are provided in 
Table II. Schools that did not administer a particular test 
are designated with “not applicable” (NA). In general, the 
problem solving and conceptual measures administered to 
students showed a consistent advantage to the Conceptual 
Problem Solving sections over the classes taught more 
traditionally. 

The problem solving test scores were significantly 
different at school C, differing by 16% [f(28) = 3.17, 
p < 0.005], with the CPS class having stronger skills at 
both selecting and applying an appropriate principle. The 
differences in problem solving scores were large but did not 
reach significance at the other two schools; school A (10%) 
and school B (15%). Each problem on the test was scored 
according to 6-7 subparts as described in the assessment 
procedures section (useful description or diagram, physics 
principle, specific application of physics, mathematical 
procedures, final units, and communicating reasoning.) 
Each part received a score of 0, 0.25, 0.5, 0.75, or 1. 
The interrater reliability for the two scorers was a Cohen’s 


TABLE II. Perfomiance on each assessment by condition and school. The Conceptual Problem Solving treatment group typically 
scored higher than the Equation Problem Solving conUol group. Values represent a percent score ± the standard error of the mean and 
stars indicate significant differences p < 0.05. 


Assessment 

School A 

SchoolB 

School C 

CPS (N = 14) 

EPS (N = 23) 

CPS (N = 8) 

EPS (N = 9) 

CPS (N = 12) 

EPS (N= 18) 

Problem solving 

0.48 ±0.10 

0.38 ± 0.05 

0.41 ± 0.08 

0.26 ± 0.06 

0.55 ± 0.04 

0.39 ± 0.03* 

Conceptual questions 3 

0.32 ± 0.04 

0.22 ± 0.02* 

0.35 ± 0.06 

0.15 ±0.02* 

0.46 ± 0.09 

0.35 ± 0.05 

Categorization 

0.47 ± 0.03 

0.45 ± 0.03 

0.46 ± 0.08 

0.35 ± 0.04 

NA 

NA 

Finding errors 

0.06 ± 0.03 

0.14 ±0.03 

0.06 ± 0.03 

0.05 ± 0.03 

NA 

NA 

Equation instantiation 

0.54 ± 0.07 

0.42 ± 0.05 

0.52 ± 0.05 

0.51 ±0.07 

0.57 ± 0.08 

0.52 ± 0.06 


a At school A the conceptual questions were free response explanations and at schools B and C they were multiple choice questions. 
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weighted kappa (quadratic squares) value of 0.72 ± 0.02 
indicating substantial agreement [36]. Each subpart was 
weighted to obtain a total score out of five points (with 
more weight placed on the physics principle and applica¬ 
tion). Scores reported in Table II represent consensus scores 
reached by the two scorers. 

The conceptual questions test showed a significant 
difference at two of the schools, a 10% difference at school 
A [f(35) = 2.54, p < 0.05] and a substantial 20% differ¬ 
ence at school B [t(15) = 3.53, p < 0.005]. Although not 
significant, school C also showed a difference of 11% 
favoring the CPS class. For the free-response version of the 
conceptual questions test administered at school A, each of 
the six questions was scored according to 2 or 3 subparts as 
0, 0.5, or 1 point. The interrater reliability for two scorers 
was a Cohen’s weighted kappa (quadratic squares) value of 
0.48 ± 0.06 indicating moderate agreement. 

The categorization test, equation instantiation test, and 
finding errors tests did not show any significant differences 
between groups. However, there was an 11% difference in 
categorization at one of the two schools that administered 
the test (2% at the other school), and a 12% difference in 
equation instantiation at one school (with 1% and 5% at the 
other schools), all differences favoring CPS. The finding 
errors test was extremely difficult for all sections at both 
schools that administered it, with almost all students 
receiving no credit for their answers. 

2. Interpretation of assessment scores 

Overall, the CPS classes performed better, but the 
differences for tests and schools varied. The assessment 
scores require a bit of interpretation, but we stress two 
points in evaluating the results. First, there were not large 
numbers of students in these classes, so the standard errors 
are sometimes large. Given the large differences in schools 
and implementation, we decided that the individual school 
data are more informative than some way of combining it 
across the schools. Second, there was great variation in the 
fidelity of the implementation. Finally, the CPS part of the 
course was quite short (less than 7 h during a semester). We 
begin by addressing each test. 

The traditional measure in physics classes, problem 
solving, showed a clear benefit for CPS across the schools 
but with much variation between students as well. The 
mean differences were 10%, 15%, and 16%, though only 
the last reach conventional levels of significance. Given the 
short amount of time used for CPS, these are consistent and 
important increases. 

The conceptual questions also showed consistent and 
important increases, with CPS advantages of 10%, 20%, 
and 11%, with the first two having a statistically significant 
difference at the 0.05 level. Note that between these two 
tests, all schools showed one statistically significant differ¬ 
ence and one sizable, if not quite significant, difference. 
The results of the problem solving and conceptual 


questions provide strong evidence for the effectiveness 
of the CPS intervention. 

The categorization test was administered at two schools 
and did not show consistent or significant differences (2% 
and 11 %). We expected that this test would be sensitive to 
the CPS intervention and were surprised the overall effects 
were not larger. We mention two issues that may have 
contributed to smaller than expected effects. First, these are 
difficult tests. Remember, the student was given a choice of 
two problems to match to the model, one of which was very 
similar superficially but was solved by a different principle 
and one of which was very dissimilar superficially but was 
solved by the same principle (see Appendix). As Table II 
shows, all groups chose the superficially similar one 
slightly more often than the similar principle one (all 
average scores are below 50%). Second, after we conducted 
this study, we have done additional work on problem 
categorization [37]. That study showed that even once 
people appreciated that principles should be used to 
categorize, they found it very difficult to determine the 
underlying principles. This test required students to deter¬ 
mine at least two underlying principles (the model and one 
other) and to overcome a strong tendency for novices to 
choose superficially similar problems. 

The finding errors test was simply too difficult to find 
differences—almost all students received no credit for their 
answers. We knew that such a test requires rather different 
skills than the CPS focus, but thought it would be 
interesting to examine if there was transfer. However, 
given the low performance, it is difficult to know whether 
the problem is the low transfer or difficulty of the particular 
problems. 

The equation instantiation test showed small nonsignifi¬ 
cant advantages for the CPS intervention (12%, 1%, and 
5%). We expected any effect of CPS here would be small, 
since it tests a more specific understanding (objects 
matched to variables) than the CPS focus, although the 
trend favors the CPS classes. It is possible that a better 
understanding of why a concept is applicable could also 
help to understand how to instantiate the particular vari¬ 
ables in the equation of that principle. 

Overall, there was a clear advantage for CPS, particu¬ 
larly, for the problem solving and conceptual questions, 
despite the little time used for the intervention and the small 
number of students in each comparison. We do recognize 
that there were unexpected difficulties with the categori¬ 
zation and finding error tests, but we believe there is a clear 
pattern of performance that favors CPS classes over tradi¬ 
tional classes. 

3. Additional assessment issues 

We also wish to address two issues brought up in 
discussions and reviews. First, although this covered a full 
semester of a course, it would be helpful to know if CPS 
understanding is not just specific to the assessments used in 
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the study. One school (school C) gives all students district¬ 
wide tests in various subjects and the teacher was able to 
give us the summaries from this CPS year and the previous 
year for his class for each problem. For the 12 conceptual 
questions asked in the test, his class for the year using the 
CPS performed 9% better across the problems [marginally 
significant, f(ll) = 1.84, p < 0.10; the CPS class also did 
better on 9 of the 12 problems, also p < 0.10 by a sign 
test]. Thus, even after the course was over, the CPS 
intervention led to some gain in district-wide tests devel¬ 
oped independently of this study. 

Second, is there any evidence that the CPS and tradi¬ 
tional classes were of equal ability at the beginning of the 
study? This was the students’ first physics course so we 
have no way to examine prior physics ability. We chose not 
to include access to various test scores as part of the assent 
or consent forms to insure greater participation, so we 
cannot look at those (and are not sure of the correlation with 
physics performance). We think ability differences are very 
unlikely to explain the CPS advantage. Recall that each 
school had different arrangements for choosing the class 
that was CPS and the one that was traditional. In school A, 
with only one physics class per year, one year was tradi¬ 
tional and the following year was CPS (this was with a very 
experienced teacher so the extra year was not likely to 
improve teaching performance). In school B, the teacher 
had two classes and randomly chose which would be CPS 
and which traditional. In school C, the teacher had one class 
and used the class of another teacher (with far more 
experience) as the traditional class. Thus, it seems very 
unlikely that the CPS advantages were due to differences in 
prior ability. 

B. Debriefing meetings 

Each teacher met with researchers at the conclusion of 
the study for a semistructured interview referred to as a 
debriefing meeting. They were asked to describe how they 
implemented conceptual problem solving in their class and 
reflect on what they thought about teaching in this way. 

1. Comments from teacher A 

Teacher A said the approach is good in that it forces 
students to think about what they are doing and why, but 
said that students at this level struggle with the math and it 
is difficult to introduce high-level concepts right from the 
start. He also felt that students had trouble developing a 
plan before actually trying to solve the problem because 
they might need to do a little trial and error before knowing 
what principle to use. His students expressed some resis¬ 
tance to the additional writing required for a strategy, but 
appreciated that it helped them make sense of the equations. 

2. Comments from teacher B 

Teacher B observed that students had the most difficulty 
with the justification component of strategy writing. 


Identifying a relevant principle was not as difficult for 
these students because they recognized what chapter or unit 
was currently being covered, and they were never tested on 
multiple concepts at the same time. For more complicated 
problems that included multiple steps, some students also 
struggled with planning the solution. 

Teacher B stated that early in the semester he found that it 
took longer to go through problems with strategy writing and 
he had some concerns that this might hinder students in the 
treatment section if they didn’t see as many problems during 
class. Despite his initial difficulties to manage classroom 
time, he later came to the decision that the additional depth 
of coverage for the CPS section was merited: 

“7 think a lot of the style I use is more equation based. So 
I think to me it probably is a pretty good thing to slow 
down a little bit and concentrate more on the concepts. 
Because it seems like it merits that, that they would get 
more out of that and maybe even doing less of the 
problems and focus a little bit more on the conceptual. 
I think also getting them to produce more of the 
conceptual too rather than me talking to them about 
it, that they’re going through and producing it on their 
own ” (teacher B). 

3. Comments from teacher C 

Teacher C indicated that prior to using this conceptual 
approach some students would just solve problems by 
writing down some “random stuff,” because they weren’t 
sure how to start, and with this framework he did not see 
this happening. He elaborated on this when describing how 
students were assessed on their use of the approach for 
several problems on a unit exam: 

“They had to write out the principle and justify their 
answer as part A, and then part B was solve it. So it was 
sort of adapting the method. Because we had done 
enough practice on it they almost all got that. And once 
they knew the principle, solving it was easy. And that was 
one of the themes that, at least 1 took away from it, was 
that the actual algebra was really easy, it’s just kind of 
knowing, looking at this problem, how do we even start it. 
Having them focus on the principle first gave them the 
confidence to know how to solve it” (teacher C). 

He observed that students were producing a higher quality 
of work and engaging in richer discussions than in the past, 
and they performed better on a district-wide physics assess¬ 
ment than in past years. Teacher C felt encouraged by 
improved student performance in the course, especially on a 
unit test near the end of the semester: 

“So I thought, well, best way to try this out is to give 
them momentum, energy, and torque all in one test and 
I was really really happy. I think it was one of the best 
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results I’ve ever seen with those concepts all mixed 
together, all jumbled up...That’s something that I’ve 
observed in virtually every math class, is that the 
students can always do any individual skill, but when 
they see all the skills all at once on like the final, or on 
an AP test, they just freeze. ” 

Teacher C stated that he agrees with the philosophy of 
the approach, because concepts and principles are essential 
for learning math and science. He was observed to make 
this goal explicit to his class through statements such as, 
“I want to change your thinking from ‘Which equation 
should I use?’ to ‘What is the concept?”’ In the near future 
he hopes to apply the conceptual approach to classes he 
teaches in mathematics. 

C. Summary of debriefings 

Each teacher had a slightly different interpretation of the 
best use for CPS in their classes. Teacher A found it helpful 
for students to use this framework to organize their thinking 
on challenging, multistep problems after they had some 
practice with simpler (traditional) problems. Despite this 
partial implementation his students saw modest gains on 
the assessments, particularly, the free-response conceptual 
questions. Teacher B used a full-fledged implementation of 
CPS during all problem solving activities in his class. He 
exclusively utilized the sample materials designed by the 
researchers, including the worksheet templates for students. 
This resulted in widespread improvement over traditional 
methods; his students showed differences of 10% or higher 
on three of the five assessments: categorization, conceptual 
questions, and problem solving. Last, teacher C viewed 
CPS as a tool to help students distinguish among multiple 
principles that could be applied to solve problems (e.g., 
energy and momentum), and as a “starting point” for 
problem solving. Once students could identify the princi¬ 
ples needed to solve problems teacher C felt that problem 
solving became more straightforward for students. His 
students made significant gains in problem solving, and 
performed higher on a district-wide assessment. 

All teachers acknowledged that it takes more time to go 
through a conceptual problem in depth, and, consequently, 
implemented CPS in ways to suit their own instructional 
needs as well as perceived pressures (e.g., to cover more 
content). Teacher A only used the full approach for one 
fourth to one third of all problems, teacher B ended up 
assigning more problems as homework for the CPS class or 
going through fewer problems, and teacher C had students 
just practice the first two steps (principle and justification) 
for several problems and only solve one or two of them 
completely. The teachers also indicated student resistance 
to having to write more, which prompted them to grade 
their use of the approach on homework or exams. Teacher 
B also modified the two-column solutions to reduce the 
amount of writing. 


IV. GENERAL DISCUSSION 

The motivation behind the Conceptual Problem Solving 
approach is to shift students away from their focus on 
equations and mathematical procedures and toward prob¬ 
lem solving approaches based on physics concepts and 
principles. We found positive effects of CPS even after 
relatively short exposures to the approach (on average, less 
than 7 h of CPS classroom instruction over a 4-month 
period). Except for one of our measures (finding errors) 
where students from all schools performed near floor, gains 
favoring the CPS-taught classes in a battery of tests that 
included both conceptual and problem solving measures 
consistently emerged. Gains varied across schools and 
measures, likely due to differing student populations and 
different implementation of the approach by the three 
teachers. 

CPS is an approach that is flexible, because it does not 
require major restructuring of the way instruction is canned 
out in high schools and because it allows teachers to 
implement it in different ways. This was confirmed by 
observing the ways in which each teacher adapted CPS to 
align with their personal teaching style and the perceived 
needs of their students. In general, the teachers who were 
more experienced (teachers A and C) used the approach 
for a subset of problems and placed greater emphasis on 
choosing among multiple principles or combining princi¬ 
ples for problems, whereas inexperienced teacher B relied 
upon researchers’ materials for all problem solving activ¬ 
ities and as a result implemented the approach with a high 
level of fidelity. More rigorous training beforehand and/or 
during the semester may have led to a closer alignment with 
researchers’ intentions for the materials. 

The realities and constraints of working in real class¬ 
rooms with real teachers created implementation chal¬ 
lenges. For example, the participating teachers’ physics 
expertise varied, from two teachers who had physics 
undergraduate degrees to one teacher trained in chemistry 
who was teaching physics out of field for only his second 
time. Pressure to cover content and time constraints also 
limited the amount of time spent on CPS and on assessing 
its impact. Finally, students tended to resist the approach 
since it required them to do significantly more prose writing 
about physics and to carry out more conceptual discus¬ 
sions, which were both difficult for students to do and time 
consuming. 

In light of these difficulties, our experiences suggest 
possible improvements to the approach. Implementing 
CPS using technology (e.g., via a web-based program that 
presents problems and students perform analyses by 
answering questions posed by the program and receiving 
conceptual feedback) would create several advantages. For 
example, students could use CPS both for in-class problem 
solving and for homework, thereby spending more time 
thinking about the application of concepts in problem 
solving. Designing the technology-based program in ways 
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where students are queried about what concepts they could 
apply and allowing them to make selections from choices 
provided would streamline the analysis and reduce student 
frustration during times where they were stuck with the 
paper-and-pencil version. This would also add flexibility 
to teachers’ implementation of the approach. Additional 
improvements to the approach include greater emphasis on 
categorization practice, as was done at school C (identify¬ 
ing appropriate concepts and principles for problems 
without solving them) and prompting students to synthesize 
across all topics, as was done at school A. 

Looking toward the future, there is a need for curricular 
materials that help students achieve better conceptual 
understanding of physics in high school, and indications 
are that CPS is one possible way of achieving this goal. 


Further study of Conceptual Problem Solving is warranted, 
in order to determine which specific elements of a teacher’s 
implementation are linked to student performance. The 
findings from this study support the goal of emphasizing 
concepts used to solve problems, so that students do not 
treat physics problem solving as a hunt for equations that 
are forgotten soon after a course is over. 
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